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Training  for  Night  Operations  -  Research  Challenges  and  Opportunities 

John  J.  Martin 
Principal  Research  Scientist 
AFRL  711HPW/RHAS 
2620  Q  Street,  Bldg  852 
WPAFB,  OH  45433 

Introduction 

For  millennia,  planners  of  military  operations  have  used  the  cover  of  darkness  to  advantage  to  prosecute  vitally 
important  missions.  Literary  accounts  tell  of  the  Greeks’  fruitless  ten-year  siege  of  Troy,  and  their  eventual 
stratagem  of  building  a  large  wooden  horse  with  a  force  of  men  hidden  inside,  leaving  it  outside  the  city  gates,  and 
ostensibly  abandoning  the  siege  and  sailing  away.  After  the  Trojans  pull  the  horse  into  their  city  as  a  trophy,  the 
hidden  force  exits  the  horse  that  night  and  opens  the  city  gates  for  the  Greek  army,  which  sailed  back  also  under 
cover  of  darkness.  More  recent  examples  include  the  WW-I1  Allied  invasion  of  Normandy,  which  began  during 
darkness,  and  night  operations  mounted  by  the  Soviet  Red  Army  to  maintain  pressure  on  over-extended  German 
forces  during  the  Battle  of  Stalingrad;  based  on  lessons  bitterly  learned,  Russian  military  doctrine  to  the  current  day 
explicitly  recognizes  the  value  of  night  operations,  particularly  in  an  offensive  role  [1].  Contemporary  examples  of 
night  operations  include  publicized  missions  in  southwest  Asia  and  eastern  Africa.  Arguably,  a  majority  of  critically 
important  missions  now  occur  when  darkness  provides  a  measure  of  covertness.  Those  who  are  properly  trained  and 
equipped  to  operate  at  night  will  possess  a  tactical  advantage  over  those  who  are  not  similarly  prepared. 

The  2009  Capstone  Concept  for  Joint  Operations  [2]  articulates  the  need  “to  effectively  and  efficiently  prepare 
future  training  audiences  ...  for  full  spectrum  operations  anywhere  in  the  world”  and  for  “a  joint  force  with 
improved  capability  and  capacity  to  operate  covertly  and  clandestinely.”  The  2010  DoD  Strategic  Plan  for  the  Next 
Generation  of  Training  [3]  states  that  "the  specific  time,  location  and  form  of  any  particular  challenge  will  be 
practically  impossible  to  predict,  at  least  in  time  to  develop  forces  specifically  for  that  threat”  and  further  states  that 
“the  long-term  objective  is  to  produce  an  immersive  training  environment  ...  [with]  sufficient  level  of  technical  and 
operational  realism.”  Given  that  full-spectrum  operations  continue  24  hours  per  day,  operational  realism  for  training 
by  definition  should  include  the  night  environment. 

A  survey  of  all  441  exhibitors  at  a  premier  training  and 
simulation  conference  [4]  in  2011  revealed  that  115  exhibited 
visuals  intended  for  use  in  training  environments,  but  only  11 
exhibitors  or  2.5%  showcased  any  visuals  which  represented  the 
night  environment,  even  if  only  still  images;  see  Figure  1.  Tools 
for  training  night  operations  may  not  necessarily  be  easy  to 
demonstrate  in  a  brightly-lit  exhibit  hall,  but  training  for  night 
operations  seems  to  receive  comparatively  less  attention  from 
the  community  responsible  for  developing  training  tools.  This 
document  attempts  to  address  this  imbalance  by  describing 
approaches  currently  used  in  night  operations  training  simulators, 
current  gaps  and  challenges,  and  research  opportunities. 

Night  Operations  Tools  and  Training  Approaches 

Compared  to  daytime  operations.  Warfighters  in  many  night  operational  domains  depend  more  heavily  on  sensor 
systems  and  technical  aids  of  various  types  for  information  upon  which  they  base  their  decisions.  Such  technical 
aids  may  include  electro-optical  imagers  such  as  night  vision  goggles  (NVGs),  airborne  forward-looking  infrared 
(FLIR),  ground-based  thermal  imagers,  laser  designators,  or  various  combinations  of  these  for  the  purpose  of 
presenting  to  the  Warfighter  some  sort  of  rendered  image  or  cueing  of  the  area  of  interest.  The  technical 
characteristics  of  these  aids,  such  as  resolution,  field  of  view,  bandwidth,  or  spectral  emission  and/or  response,  affect 
the  user’s  perception  of  the  environment,  so  it  is  important  that  such  technical  characteristics  are  credibly  duplicated 
in  the  training  environment  so  personnel  can  understand  them  in  preparation  for  maximally  benefitting  from  them 
while  minimizing  effects  associated  with  their  limitations. 


Figure  1.  2011  Training  and  Simulation 
Conference  Exhibits  and  Visuals  by  Type 
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Training  for  night  operations  currently  occurs  most  commonly  on  live  ranges.  However,  due  to  legal,  ethical  and/or 
practical  constraints,  live  range  opportunities  and  exposure  to  some  effects  essential  for  trainees’  experience  and 
currency  may  not  be  available.  Training  gaps  can  result,  at  the  very  least  leaving  Warfighters  inadequately 
prepared;  some  gaps  have  been  implicated  in  night  mishaps  resulting  in  loss  of  life  [5],  If  properly  implemented, 
training  simulator  environments  may  provide  exposure  to  many  such  effects  and  address  some  of  these  gaps. 

Credible  training  simulator  environments  for  different  domains  require  different  types  and  quantities  of  entities  and 
effects.  For  example,  some  effects  such  as  the  appearance  of  missile  launches,  appropriate  for  creating  credible 
training  for  night  air  operations,  will  differ  from  effects  such  as  the  appearance  of  vehicle  lights  appropriate  for 
training  night  surface  operations.  Based  on  the  origin  of  some  of  these  effects  in  the  real  world,  and  the  domain  and 
type  of  training  to  be  accomplished,  a  stimulate  approach  to  the  night  training  environment  may  be  the  method  of 
choice;  for  others,  a  simulate  approach  may  be  the  best  or  only  option  available.  Both  approaches  are  described  in 
this  document.  The  underlying  assumption  for  both  approaches  is  that  credible-looking  simulated  scene  imagery 
visible  to  the  unaided  eye  is  projected  or  otherwise  presented  on  one  or  several  screens,  and  that  for  practical  and 
economic  reasons  a  training  environment  (including  projectors)  usable  for  both  day  and  night  training  is  desirable. 

All  simulator  environments  for  training  night  operations  are  just  that:  simulations.  Either  real  or  simulated  imaging 
devices  may  be  used  in  such  environments.  Simulated  imagery  that  “looks  real’’  is  rendered  (e.g.,  computer¬ 
generated)  from  stored  scene  file  models,  using  either  a  subjective  or  a  deterministic  approach.  In  a  subjective 
approach,  imagery  captured  from  sensor(s)  under  a  variety  of  representative  real-world  conditions  is  used  as  a 
reference  while  pre-adjusting  the  rendering  system,  often  with  guidance  from  a  subject  matter  expert  (SME),  until 
the  imagery  subjectively  “looks  about  right”  (SLAR).  In  a  deterministic  approach,  the  real  environment,  sensor(s), 
image  generator  (IG)  and  display  system  have  been  measured  and  physically  modeled  beforehand.  Data-based 
spectral  emission  characteristics  of  real-world  surfaces  or  objects  as  well  as  atmospheric  effects  then  quantitatively 
determine  the  appearance  of  the  sensor  display  in  the  training  environment;  e.g.,  there  is  end-to-end  quantitative 
traceability.  The  fidelity  of  imagery  may  be  a  key  consideration  when  simulator  certification  is  required  for 
personnel  to  gain  credit  for  training  occurring  in  the  simulator,  which  potentially  may  affect  choosing  between  a 
SLAR  or  a  deterministic  approach.  Accurate  rendering  of  imagery  from  a  newly-introduced  sensor  having  unique 
characteristics  may  be  a  matter  of  plugging  its  performance  model  into  a  deterministic  system,  whereas  significant 
iterative  tweaking  with  SME  guidance  may  be  required  to  approach  the  “new  SLAR”  in  a  SLAR  system. 

“Stimulate  Approach”  Training  Environments 

In  a  stimulate  approach  night  training  environment,  the  same  night 
vision  goggles  (NVGs)  used  in  the  operational  environment  are 
used  to  view  simulated  scene  imagery  presented  on  the  screen(s)  in 
the  training  environment;  the  stimulate  approach  thus  is  a  sensor- 
in-the-loop  configuration  with  NVGs  stimulated  by  the  same  scene 
imagery  viewed  by  the  unaided  eye.  The  trainee  simply  aims  their 
NVG  at  any  area  of  interest  within  the  entire  scene  presented  on 
the  screen.  Use  of  real  NVGs  in  the  training  environment  offers 
the  advantage  of  absolute  physical  fidelity  of  the  NVG,  and  avoids 
the  need  for  video  cables  or  other  tethers  that  are  not  present  in  the 
operational  environment.  Sensors  other  than  NVGs  generally 
cannot  be  used  in  stimulate  approach  environments. 

The  intensifier  tubes  in  current  NVGs  respond  to  wavelengths 
between  approximately  370  and  930  nanometers  (nm),  as  shown  in 
Figure  2.  Minus  blue  filters  are  incorporated  in  NVGs  used  for 
airborne  applications,  limiting  their  response  to  wavelengths 
between  roughly  650nm  (deep  red)  and  930nm  to  allow  them  to  be 
used  in  a  cockpit  without  being  adversely  affected  by  illuminated 
cockpit  displays.  Some  NVGs  used  for  ground  applications  do  not 
incorporate  filtering  and  thus  may  respond  to  wavelengths  across 
the  entire  radiant  sensitivity  band.  In  either  case,  in  order  to  be 
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Figure  2.  Radiant  Sensitivity  of  Intensifiers 
Currently  Used  in  Night  Vision  Goggles 
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sensed  and  intensified,  the  scene  imagery  must  exhibit  spectral  emission  that  overlaps  the  band  of  NVG 
response;  this  is  known  as  NVIS  radiance.  Both  the  amount  of  overlap  and  the  proportion  of  NVIS  radiance  to 
visible  energy  are  functions  of  -  and  will  differ  based  upon  -  the  projector/display  technology  and  NVGs  used. 

NVGs  are  very  sensitive  to  any  light  within  their  band  of  radiant  sensitivity.  The  intensifier  tubes  in  NVGs  have  a 
maximum  gain  (e.g.,  light  amplification  factor)  of  approximately  50,000,  and  incorporate  circuitry  that 
automatically  reduces  the  gain  as  ambient  light  within  their  band  of  radiant  sensitivity  increases  above  a  level 
roughly  equivalent  to  quarter  moon;  as  a  result,  the  intensified  image  luminance  is  not  allowed  to  exceed  an  upper 
limit  of  approximately  4  foot-Lamberts  averaged  across  the  entire  field  of  view.  Intensifier  gain  control  is  a  function 
of  the  total  input  across  the  NVG  instantaneous  field  of  view,  regardless  of  the  source;  as  a  result,  only  one  or  a  few 
small  but  bright  light  sources  may  be  sufficient  to  reduce  gain  and  affect  the  entire  image.  As  gain  decreases,  the 
luminance  of  other  objects  of  interest  appearing  in  the  intensified  image  decreases  and  they  become  harder  to  see. 
Therefore,  strict  light  discipline  must  be  observed  in  stimulate  approach  training  environments  so  projected  scene 
imagery  is  the  only  significant  source  of  energy  stimulating  the  NVGs.  Light  from  non-scene  sources  such  as 
computer  monitors,  cockpit  displays,  equipment  status  lights,  room  lighting,  exit  signs,  windows,  etc.,  that  may  fall 
within  the  NVG  field  of  view  must  be  extinguished,  blocked,  filtered,  or  otherwise  controlled  so  that  it  does  not 
affect  the  intensifier  gain  and  NVG  image. 

The  dynamic  range  of  energy  present  in  the  night  environment  due  to  natural  illumination  conditions  alone  spans 
three  orders  of  magnitude  from  overcast  starlight  to  full  moon.  When  considering  artificial  light  sources  and 
illuminated  objects,  such  as  bright  urban  area  lighting,  laser  pointer  or  designator  beams,  or  munitions  effects  that 
also  would  plausibly  be  present  in  operational  environments,  this  span  of  dynamic  range  can  easily  double  or  even 
triple  to  encompass  a  total  of  nine  or  ten  orders  of  magnitude.  Ideally  a  stimulate  approach  training  environment 
would  be  capable  of  presenting  scenes  replicating  the  dynamic  range  present  under  any  operational  conditions,  such 
that  the  NVGs  would  be  stimulated  to  produce  imagery  matching  that  which  they  would  produce  in  those 
operational  conditions.  Projector  technology  offering  a  contrast  ratio  of  up  to  seven  orders  of  magnitude  [6]  has 
recently  been  introduced,  but  at  the  current  time  no  projector  technology  is  available  that  has  a  dynamic  range 
capability  wide  enough  to  match  that  of  all  real  world  conditions.  When  operating  projectors  near  the  extreme  lower 
end  of  the  brightness  gamut  to  present  a  night  scene,  few  brightness  levels  may  remain  and  imagery  may  appear 
unrealistic.  In  such  cases  a  neutral  density  filter  may  be  introduced  in  front  of  a  projector  lens  to  allow  it  to  operate 
at  more  normal  settings  and  restore  most  or  all  brightness  levels.  Similar  effects  may  be  achieved  with  a  front 
projection  screen  having  low  gain,  although  at  the  expense  of  day  scene  luminance.  In  addition  to  neutral  density 
filtering,  selective  color  filtering  also  may  be  introduced  in  front  of  a  projector  to  achieve  credible-appearing  night 
scenes  from  both  the  NVG-aided  and  unaided  perspectives.  While  easy  to  implement,  filters  are  attenuators  and 
their  use  will  reduce  the  amount  of  energy  reaching  the  screen,  potentially  reducing  the  effect  of  the  projected  scene 
on  NVG  gain.  As  a  result,  stimulated  NVG  imagery  duplicating  that  seen  in  the  real-world  may  be  achievable  only 
within  limited  dynamic  ranges  and  lower  illumination  conditions. 

Viewed  scene  content  consists  of  both  surface  areas  and  point  light  sources.  The  spectral  emission  and  reflectivity 
characteristics  of  many  surfaces  and  sources  have  been  measured  and  modeled,  and  significant  work  has  been 
performed  in  developing  methodologies  to  translate  known  radiance  and  reflectivity  values  of  real-world  objects 
under  defined  conditions  into  known  luminance  values  (e.g,  physics-based  deterministic  renderings)  of  scene 
elements  in  NVG  intensified  imagery  [7].  Significant  work  also  has  been  done  in  capitalizing  on  capabilities  of 
commonly-employed  projectors  (e.g.,  addition  of  filtering)  to  support  affordable  solutions  that  provide  imagery 
which  appears  credible  through  NVGs,  but  challenges  remain  in  achieving  rendering  of  extended  surfaces  and  bright 
point  sources  that  simultaneously  appear  credible  to  the  unaided  eye. 

Bright  point  light  sources,  such  as  streetlamps  and  vehicle  lights,  are  significant  contributors  to  the  wide  dynamic 
range  found  in  the  night  operational  environment,  and  point  sources  commonly  induce  formation  of  a  bright  disc 
(halo)  that  surrounds  each  light  point  in  an  NVG  image.  Such  halos  can  be  important  visual  factors  when 
prosecuting  a  mission  and  therefore  should  be  included  in  a  night  training  environment.  Some  recently-introduced 
projector  technology  [6],  although  expensive,  can  present  light  points  that  induce  realistic-appearing  halos  in  an 
NVG  image.  However,  most  commonly-employed  projectors  are  not  capable  of  presenting  light  points  with 
sufficient  intensity  to  induce  halos  of  realistic  density,  particularly  if  projector  filtering  is  used.  Modeled  halos  may 
be  generated  to  surround  point  light  sources  in  the  scene  rendered  by  the  IG.  Because  visible  halos  normally  do  not 
surround  bright  point  light  sources  in  the  real  world,  modeled  halos  may  be  rendered  only  in  dark  red  to  minimize 
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their  visibility  to  the  unaided  eye  while  exploiting  the  high  sensitivity  of  NVGs  to  red  wavelengths.  Healthy  foliage 
which  appears  dark  to  the  unaided  eye  also  can  appear  bright  in  an  NVG  intensified  image;  as  long  as  the  scene 
luminance  remains  at  a  low  level  where  human  color  perception  is  poor,  foliage  may  similarly  be  rendered  in  dark 
red.  Projectors  recently  have  been  developed  which,  in  addition  to  visible  channels,  include  a  near-infrared  channel 
dedicated  to  stimulating  NVGs.  Such  projectors  facilitate  presentation  of  imagery  that  appears  realistic  both  through 
NVGs  and  to  the  unaided  eye,  but  cost  may  become  a  limiting  factor  if  multiple  such  projectors  are  required  to  fill  a 
large  screen  surface  area  (e.g.,  a  dome)  with  imagery. 

In  an  operational  night  environment  the  viewing  distance  between  an  NVG  and  the  area  of  interest  usually  is  infinity 
in  optical  terms,  but  in  a  training  environment  the  distance  between  the  NVG  and  the  screen) s)  being  viewed  is  very 
short  in  comparison  (e.g.,  often  one  or  two  meters).  At  close  focus  distances  the  objective  lenses  of  NVGs  exhibit  a 
very  shallow  depth  of  field;  when  the  lenses  are  adjusted  for  sharply-focused  imagery  at  one  close  distance,  imagery 
will  be  defocused  -  potentially  severely  -  at  slightly  shorter  and  longer  viewing  distances.  For  example,  with  an 
NVG  focused  for  sharpest  acuity  (Snellen  20/25)  at  a  viewing  distance  of  one  meter,  an  increase  of  30  centimeters  in 
viewing  distance  will  cause  NVD-aided  visual  acuity  (without  refocusing)  to  degrade  to  worse  than  20/100.  A 
pinhole  aperture,  functioning  as  an  f-stop,  may  be  attached  on  the  front  of  an  NVG  objective  lens  to  increase  the 
depth  of  field;  however,  the  scene  luminance  then  must  be  raised,  potentially  compromising  its  realism  as  seen  by 
the  unaided  eye.  An  added  pinhole  aperture  also  constitutes  an  optical  element  not  part  of  the  original  objective  lens 
design,  and  it  will  degrade  the  NVG  image  resolution.  NVG  depth  of  field  increases  and  defocus  effects  are  reduced 
in  environments  with  longer  viewing  and  corresponding  focus  distances,  blit  longer  viewing  distances  and  larger 
physical  sizes  of  training  environments  may  equate  to  higher  costs.  Therefore,  intended  training  domains  and 
scenarios,  the  optical  characteristics  of  equipment  to  be  used,  and  freedom  of  movement  of  trainees  become  tradeoff 
factors  that  must  be  examined. 

Imaging  equipment  used  in  the  military  operational  environment  typically  has  excellent  optics.  When  such 
equipment  is  used  in  a  stimulate  approach  training  environment,  the  net  resolution  obtained  by  the  trainee  typically 
is  limited  by  the  resolution  of  the  scene  imagery  presented  on  the  screen,  which  itself  is  a  function  of  the 
projector(s),  IG(s)  and  imagery  database.  Use  of  equipment  having  magnifying  optics  (e.g.,  binoculars)  is  not 
feasible  in  stimulate  approach  environments  because  the  scene  imagery  projected  on  the  screen  will  appear  in  the 
eyepieces  simply  as  a  magnified  view  of  a  small  number  of  scene  pixels;  viewed  resolution  will  decrease  as  the 
inverse  square  of  the  magnification  factor. 

The  foregoing  discussion  has  applied  to  unaided  visible  and  NVG-aided  imaging,  but  thermal  imagers  such  as  FLIR 
systems  on  aircraft  and  portable  systems  used  by  ground  personnel  also  are  employed  in  the  operational 
environment.  Use  of  an  operational  thermal  imager  in  a  stimulate  approach  training  environment  would  require  that 
all  objects  rendered  in  the  scene  include  realistic  thermal  signatures  in  the  band  to  which  the  thermal  imager 
responds.  Models  of  thermal  signatures  exist  for  many  scenes  and  objects,  but  practical  means  currently  do  not  exist 
for  projecting  or  otherwise  presenting  thermal  imagery  on  the  screen(s)  at  a  resolution  useful  for  training.  Thermal 
imaging  equipment  with  magnifying  optics  also  is  not  usable  in  stimulate  approach  environments  for  the  reasons 
previously  given.  Physically  emulated  (e.g.,  “feels  like,  looks  like”)  thermal  imagers  presenting  simulated  imagery 
thus  are  the  only  option  currently  available.  The  aim  of  such  imagers  must  be  accurately  tracked  in  real-time  so  the 
simulated  imagery  is  registered  (e.g.,  aligned)  with  other  imagery  simultaneously  presented  to  the  trainee.  A  cable 
or  wireless  data-link  is  required  to  transmit  video  of  simulated  imagery  to  the  emulated  device.  The  potential 
classification  level  of  information  in  the  training  environment  should  be  an  early  consideration  when  choosing  the 
technology  for  transmitting  signals  to  emulated  devices;  depending  on  its  characteristics,  gaining  approval  for  use  of 
a  wireless  data-link  to  transmit  classified  information  may  prove  to  be  challenging. 

Laser  pointers  and  designators  are  commonly  used  in  the  ground  and  close  air  support  environment,  so  their  effects 
also  should  be  realistically  represented  in  the  training  environment.  Operational  laser  devices  have  output  power 
levels  appropriate  for  targets  at  distances  of  up  to  several  kilometers.  Such  lasers  are  not  eye  safe  and  cannot  be 
used  in  indoor  training  environments.  Physically  emulated  laser  devices  and  simulation  of  their  effects  thus  are  the 
only  option  available.  The  position  and  aim  of  the  emulated  laser  device  must  be  accurately  tracked  in  real-time, 
and  a  simulation  of  its  beam  with  foreshortening  appropriate  from  the  trainee’s  viewing  perspective  must  be 
projected  on  the  display  screen  at  a  wavelength  to  which  NVGs  respond,  if  appropriate  for  the  laser  device  involved. 
The  simulated  beam  also  must  be  presented  in  the  eyepiece  of  the  simulated  thermal  imager  or  designator  equipment 
if  appropriate. 
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“Simulate  Approach”  Training  Environments 


In  a  night  training  environment  using  a  simulate  approach,  all  devices  (including  NVGs)  employed  by  trainees  are 
emulated  (physically  reproduced)  and  all  imagery  is  simulated.  Imagery  is  created  either  through  deterministic 
(physics-based)  rendering,  or  through  subjective  adjustment  of  the  renderings  (the  SLAR  approach),  both  as 
previously  discussed.  For  highest  training  fidelity  the  physical  configuration  of  the  emulated  equipment  should 
match  that  of  operational  equipment,  but  means  are  required  for  conveying  video  and/or  signals  to  and  from  the 
emulated  equipment.  The  aim  and  orientation  of  all  emulated  imaging  equipment  also  must  be  accurately  tracked  in 
real-time  so  the  simulated  sensor  imagery  is  accurately  aligned  with  unaided  scenes  simultaneously  presented  to  the 
trainee.  Provisions  for  signal  transmission  and  real-time  tracking  will  add  physical  features  such  as  cables  or 
modules  not  present  on  operational  equipment;  such  features  can  be  a  concern  for  head-worn  equipment  such  as 
NVGs,  and  should  be  as  unobtrusive  as  possible.  The  anticipated  classification  level  of  information  in  the  training 
environment  needs  to  be  an  early  consideration  when  choosing  the  technology  used  for  transmitting  signals  to  and 
from  emulated  devices;  information  security  will  trump  physical  fidelity. 

Emulated  imaging  equipment  usually  does  not  require  objective  optics,  so  depth  of  field  typically  is  not  a  factor  and 
the  distance  between  the  equipment  and  the  simulator  screen  typically  has  no  effect  on  image  focus.  Similarly, 
because  the  emulated  equipment  is  insensitive  to  light,  strict  light  discipline  need  not  be  observed  in  stimulate 
approach  training  environments.  Light  originating  from  non-scene  sources  need  not  be  extinguished  or  blocked, 
although  it  should  be  well-controlled  to  avoid  creating  visual  distractions  to  trainees. 

Because  only  simulated  imagery  is  presented  to  trainees  in  simulate  approach  training  environments,  dynamic  range 
effects  for  any  night  environment  may  be  rendered  in  the  simulated  NVG  imagery;  the  dynamic  range  limitations  of 
projectors  become  a  moot  point.  Significant  research  in  physics-based  simulation  led  to  development  of  the  Night 
Vision  Training  System  (NVTS)  [8]  which  was  enabled  by  the  SensorHost  system  [9]  that  performed  all  physics  and 
NVG-specific  computations  for  the  IG.  NVTS  was  successfully  demonstrated  in  an  F-16C  training  simulator  at  Air 
Force  Research  Laboratory  in  2003.  While  SensorHost  functions  initially  were  performed  using  separate  equipment 
connected  to  the  IG,  more  recently  the  SensorHost  functions  have  been  integrated  directly  into  IGs. 

Simulator  Fidelity 

In  the  psychology  literature,  Hayes  and  Singer  [10:  1]  state  that  fidelity  in  the  context  of  training  systems  may  be 
defined  as  the  level  of  realism  that  a  simulation  presents  to  the  learner,  this  concept  being  an  integral  component  in 
simulation  because  it  defines  “how  similar  a  training  situation  must  be,  relative  to  the  operational  situation,  in  order 
to  train  most  efficiently.”  They  summarize  fidelity  as  “. .  .the  degree  of  similarity  between  the  training  situation  and 
the  operational  situation  which  is  simulated.  It  is  a  two  dimensional  measurement  of  this  similarity  in  terms  of:  (1) 
the  physical  characteristics,  for  example  visual,  spatial,  kinesthetic,  etc.;  and  (2)  the  functional  characteristics,  for 
example  the  informational,  stimulus,  and  response  options  of  the  training  situation.”  [10:  50]  In  a  related  vein, 
Gagne  [11:  99]  states,  “close  simulation  has  the  aim  of  insuring  high  validity  to  the  task  presented  to  the  trainer.  It 
is  not  quite  as  easy  as  it  sounds,  though,  to  produce  a  highly  valid  measurement  of  performance  by  exact  simulation 
of  the  operational  environment.”  The  primary  reason  for  these  seemingly  counterintuitive  findings  lies  with  a 
fundamental  understanding  of  how  humans  perceive  and  process  sensory  information.  A  discussion  of  human 
perception  and  processing  of  sensory  information  is  beyond  the  scope  of  this  document,  but  the  end  goal  of  training 
systems  is  increased  effectiveness,  so  it  seems  fair  to  argue  that  training  systems  should  be  developed  to  maximize 
their  effectiveness,  not  necessarily  their  fidelity  (for  which  no  published  objective  methods  of  measurement  appear 
to  exist). 

A  simulator  does  not  need  to  provide  an  exact  representation  of  the  real  world  in  order  to  provide  effective  training. 
In  fact,  some  departure  from  exact  realism  may  be  necessary  in  order  to  provide  the  most  effective  training  [10:  15; 
11:  101].  Rendering  systems  based  entirely  on  physics-based  models  and  having  deterministic  end-to-end 
traceability  can  provide  exact  representation  (e.g.,  fidelity),  but  such  systems  may  be  unnecessary  for  effective 
training  and  may  be  very  difficult  to  implement  in  practice,  particularly  in  the  case  of  multiple  networked  systems 
encompassing  different  night  domains  and  utilizing  different  IGs  and  imagery  databases.  This  leads  to  the 
hypothesis  that  something  less  than  full  physics-based  rendering,  e.g.,  quasi-deterministic  rendering,  could  provide 
simulation  fidelity  sufficient  for  effective  training  even  though  it  would  not  provide  an  exact  duplication  of  the 
operational  environment. 
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Standards  Research  Opportunities 

A  set  of  standard  reference  surfaces  and  objects,  lighting  conditions,  appropriate  image  characteristics  for 
commonly-utilized  sensor  devices  such  as  NVGs  and  thermal  imagers,  and  measurement  techniques  could  enable 
adjustment  or  calibration  of  any  IG  and  the  associated  display  system  to  achieve  quasi-deterministic  renderings.  An 
example  concept  might  be  the  SMPTE  color  chart  [12;  13]  used  during  the  setup  of  video  systems.  Ideally,  such  a 
set  of  standards  would  be  universally  applicable  to  training  simulators  for  all  types  of  night  operations.  It  seems 
reasonable  to  assume  that  such  a  set  of  standards  would  need  to  be  bounded  to  some  limited  quantity  that  is  easily 
dealt  with,  to  facilitate  adoption  and  utilization  across  simulator  platforms  and  services.  Given  a  bounded  quantity, 
interpolation  would  be  required,  and  the  validity  of  the  interpolations  would  depend  on  selection  of  the  references 
and  the  chosen  intervals  between  them.  Research  is  needed  to  define  such  standard  references  and  intervals, 
informed  by  those  representation  effects  identified  by  the  operational  community  as  essential  for  training  night 
operations.  Such  research  also  needs  to  define  the  minimum  dynamic  range  in  the  night  scene  that  still  will  allow 
immersive  training  for  any  night  mission  domain. 

In  distributed  mission  operations  (DMO),  live,  virtual  and  constructive  environments  are  data-linked  together  for 
training.  Multiple  simulator  environments,  each  exposing  trainees  to  virtual  and/or  constructive  components  of  the 
overall  DMO,  may  be  networked  together  at  one  time,  and  it  is  possible  that  each  of  those  simulator  environments 
may  employ  a  different  type  of  IG,  each  with  its  own  proprietary  protocols.  The  common  image  generator  interface 
(CIGI)  [14]  is  an  open-source  non-proprietary  simulation  protocol  for  communication  between  a  host  device  and  an 
IG,  intended  to  promote  interoperability  among  IGs.  CIGI  is  a  useful  foundation  for  advancing  the  standardization 
of  commands  between  hosts  and  IGs,  but  it  only  goes  so  far,  and  does  not  define  IG  functional  requirements,  e.g., 
what  happens  in  an  IG’s  rendering  process  and  the  resulting  appearance  of  the  presented  imagery.  For  DMO  in 
which  a  simulate  approach  environment  may  be  linked  via  network  with  a  different  environment  using  a  stimulate 
approach  and/or  a  different  IG,  common  standards  and  methodologies  are  needed  to  ensure  that  the  renderings  of 
entities  and  effects  visible  to  trainees  in  each  environment  appear  credible  to  those  trainees  from  their  particular 
point  of  view,  and  are  matched  (e.g.,  synchronized)  in  luminance,  radiance,  temporal  and  spatial  aspect  appropriate 
to  individual  perspectives  and  locations,  regardless  of  the  IG  type(s)  employed  at  each  location.  Credible  rendering 
of  weather  effects  on  sensor  performance  for  each  particular  training  environment’s  point  of  view  also  is  needed. 
Such  common  standards  would  be  a  significant  enabler  for  quasi-deterministic  renderings  across  distributed  mission 
training  facilities.  Research  is  needed  to  define  such  standards. 

Representation  Effects  for  Night  Training 

The  joint  terminal  attack  controller  (JTAC)  community  has  identified  training  for  night  close  air  support  (CAS)  as 
critical  for  tactical  success  and  operations.  Based  on  current  doctrine  and  projected  operations  that  will  include  both 
covert  and  force-on-force  engagements  between  near-peers,  the  JTAC  community  has  identified  the  following 
representation  effects  (assuming  renderings  of  terrain,  vegetation,  buildings  and  vehicles)  as  essential  in  training 
environments  for  CAS  at  night: 

•  Parachute  flares 

•  Laser  marker  and  designator  beams  (ground-  and  air-based) 

•  Ambient  illumination  (moon  phase  and  angle) 

•  Artillery  flashes  (outbound  and  incoming)  and  tracers 

•  Cultural  lighting,  vehicle  lights,  fires,  explosions 

•  Dust  and  weather 

•  Night  vision  (including  point  source  halo  effects) 

•  Thermal  targets  imaging 

•  Strobes  (friendly  forces) 

•  Remotely  Operated  Video  Enhanced  Receiver  (ROVER)  video  display 

In  addition  to  these  effects  or  their  close  equivalents,  the  following  representations  also  are  appropriate  for  training 
the  air  component  of  night  CAS: 

•  Rocket-propelled  grenades  and  missile  launches 

•  Aircraft  external  lighting  (unaided  and  NVG-aided) 
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Research  is  needed  to  determine  whether  any  representation  effects  in  addition  to  the  above  are  essential  to 
constitute  a  list  that  could  be  universally  applicable  across  services  and  across  all  types  of  night  operations. 


Wide  Dynamic  Range  Technical  Implementation 

To  this  point  the  discussions  in  this  document  have  focused  on  approaches,  limitations  and  associated  research 
opportunities  in  creating  credible  environments  for  training  night  operations.  It  also  is  appropriate  to  discuss  means 
of  achieving  wide  dynamic  ranges  in  stimulate-approach  environments. 

Stimulate-approach  environments  for  training  night  urban  operations  should  be  capable  of  stimulating  NVGs  such 
that  intensifier  gain  varies  from  a  maximum  (e.g.,  50,000)  in  no-moon  austere  area  conditions  to  a  minimum  (e.g., 
100  or  less)  in  urban  areas  with  streetlamps  and  vehicle  lights.  Dynamic  range  limits  of  current  projectors  make 
them  incapable  of  inducing  such  realistic  gain  variations.  In  theory,  with  real-time  tracking  of  the  NVG  aim,  the 
brightness  of  projected  scenes  containing  bright  sources  could  be  modulated  to  emulate  the  gain  response  of  NVGs 
when  exposed  to  such  sources  in  the  real  world.  However,  modulation  of  scene  brightness  based  on  tracking  would 
be  effective  only  for  one  person  at  a  time,  which  could  be  problematic  in  a  multi-person  environment.  Such  tracking 
also  would  require  attachment  of  ancillary  equipment  to  the  NVGs  which  would  negate  some  of  the  advantages  of 
using  real  NVGs.  Given  that  the  same  display  projectors  should  be  usable  for  both  day  and  night  scenes  for  sake  of 
affordability,  but  projectors  currently  are  not  capable  of  producing  scene  dynamic  ranges  that  encompass  all  effects 
needed  for  credible  rendering  of  the  night  environment  for  training,  separate  means  are  desirable  to  create  bright 
point  light  sources  that  appear  credible  to  the  unaided  eye  while  also  inducing  appropriate  gain  variation  and  halo 
effects  in  NVG  imagery. 

Inexpensive,  eye-safe  (e.g.,  Class  1  or  Class  2)  laser  pointers  can  project  bright  light  points  that  induce  realistic  gain 
reduction  and  halo  effects  when  viewed  by  NVGs.  By  combining  the  beams  from  red  and  green  lasers  with  a  simple 
optical  combiner,  a  brilliant  amber  point  results  that  appears  credible  as  a  parachute  flare,  vehicle  headlamp  or 
streetlight  to  the  unaided  eye.  Such  a  combined  laser  approach  has  been  successfully  demonstrated  in  a  JTAC 
training  environment  at  71 1HPW/RHA.  Selective  use  of  only  a  red  or  green  laser  also  can  allow  realistic  simulation 
of  colored  tracers  used  in  actual  field  conditions.  The  output  power  of  such  lasers  may  be  modulated  through  simple 
current  limiting  of  the  electrical  power  source.  Steering 
of  the  laser  beam  may  be  accomplished  by  directing  the 
output  beam  from  the  combiner  optics  into  an  X-Y 
mirrored  galvanometer  assembly;  see  Figure  3.  A  “point 
source  engine”  thus  can  be  built  using  eye-safe  red  and 
green  lasers,  an  optical  beam  combiner,  and  an  X-Y 
mirrored  galvanometer  assembly.  Available  X-Y 
galvanometer  movement  assemblies  [15],  about  the  size 
of  an  orange  and  with  low  mass  and  inertia,  are  capable  of 
high  speed  and  accuracy  appropriate  for  credible 
rendering  of  the  movement  effects  of  repetitious  artillery 
rounds  as  seen  from  operationally  realistic  distances. 

Position  encoding  of  the  X  and  Y  galvanometer  axes, 
linked  to  the  IG  via  a  closed  loop  control  system,  can 
allow  accurate  alignment  of  the  projected  laser  point  with 
appropriate  entities  in  the  projected  imagery. 

Either  a  vector  or  raster  scan  approach  may  be  used.  When  a  vector  approach  is  used,  each  bright  light  point 
appearing  in  the  scene  at  any  given  time  requires  one  point  source  engine;  with  a  raster  scan  approach,  one  point 
source  engine  can  project  multiple  bright  light  points  in  the  scene.  The  number  of  bright  light  points  required  for 
credible  training  scenarios  will  differ  depending  on  the  domain  and  mission  type.  The  minimum  required  number  of 
point  source  engines  thus  will  depend  on  the  domain  and  mission,  the  scanning  approach  used,  the  size  of  the 
projection  screen  in  the  training  environment,  the  distance  and  positioning  of  each  engine  with  respect  to  the  screen, 
and  the  angular  coverage  (“footprint”)  of  each  engine.  Research  is  needed  to  determine  the  minimum  number  of 
bright  point  lights  that  simultaneously  need  to  appear  in  a  scene  to  achieve  a  credible  training  representation 
(scenario  dependent). 
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The  ability  of  a  raster  approach  to  project  bright  light  points  with  sufficient  energy  to  induce  realistic  gain  and  halo 
effects  in  NVGs  is  open  to  question.  Assuming  use  of  eye-safe  lasers  in  a  point  source  engine  running  in  raster 
mode  at  60Hz  refresh  rate,  serving  a  screen  area  of  3.5m2,  and  with  a  projected  bright  light  point  located  to  a  spatial 
precision  of  1cm  and  filling  1cm2  of  the  screen  area,  the  integrated  energy  per  unit  time  per  cm2  of  screen  area  will 
be  more  than  six  orders  of  magnitude  lower  than  the  constant-on  energy  from  a  vectored  light  point  originating  from 
the  same  laser  sources.  Expressed  another  way,  the  raster  will  dwell  on  inactive  screen  area  more  than  99.99%  of 
the  time.  Research  is  needed  to  determine  whether  galvanometer  speed  is  sufficient  to  avoid  distracting  flicker 
effects  with  a  raster  scan  approach,  and  whether  eye-safe  lasers  provide  sufficient  energy  to  induce  gain  variation 
and  halo  effects  in  NVGs.  The  perceived  fidelity  of  raster  and  vector  approaches  also  should  be  compared.  If 
flicker  effects  with  a  raster  approach  prove  problematic  or  spot  energy  is  insufficient  to  induce  gain  and  halo  effects 
in  NVGs,  then  a  vector  approach  may  be  the  only  option.  In  this  case  the  minimum  number  of  bright  point  lights 
simultaneously  appearing  in  a  scene  defines  the  number  of  point  source  engines  required.  With  artful  planning  of 
training  scenarios,  the  quantity  of  bright  light  points  simultaneously  appearing  on  the  screen(s),  and  the 
corresponding  point  source  engines,  could  be  limited  to  a  manageable  number,  e.g.,  perhaps  four  or  five  total. 

Of  the  essential  representation  effects  previously  listed,  the  parachute  flares,  artillery  flashes  and  tracers,  cultural 
lighting,  vehicle  lights,  fires,  explosions,  NVG  halo  effects,  and  strobes  may  be  created  using  an  overlay  of  the  eye- 
safe  laser  points  and  conventional  projected  imagery  or  by  the  lasers  alone.  With  appropriate  calibration  using  the 
standards  previously  discussed,  the  resulting  total  dynamic  range  presented  in  the  training  environment  could 
approach  or  match  that  existing  under  most  reasonable  real-world  night  conditions. 

Summary  of  Research  Needs  and  Opportunities 

The  following  is  a  summary  of  research  needs  and  opportunities,  as  previously  described  in  this  document, 
appropriate  for  creating  credible  training  environments  for  night  operations. 

•  Define  domains,  platforms/Mission  Design  Series,  and  mission  types  (in  addition  to  JTAC)  for  which  data 
should  be  collected  on  night  representations  required  for  credible  realism  and  training  experiences.  Include 
in  Mission-Essential  Competency  analyses. 

•  Define  the  minimum  dynamic  range  in  the  night  scene  that  still  will  allow  immersive  training  for  any  night 
mission  domain. 

•  Define  a  set  of  standard  reference  surfaces  and  objects,  lighting  conditions,  appropriate  image 
characteristics  for  commonly-utilized  sensor  devices  such  as  NVGs  and  thermal  imagers,  and  measurement 
techniques  to  enable  adjustment  or  calibration  of  any  IG  and  the  associated  display  system  to  achieve 
quasi-deterministic  renderings.  Also  define  the  intervals  between  such  standard  references.  (The  SMPTE 
color  bar  chart,  as  used  for  calibration  of  color  video  systems,  might  serve  as  an  example  concept.) 

•  Define  common  rendering  standards,  to  ensure  that  the  renderings  of  entities  and  effects  visible  to  trainees 
in  each  environment  appear  credible  to  those  trainees  from  their  particular  point  of  view,  and  are  matched 
(e.g.,  synchronized)  in  luminance,  radiance,  temporal  and  spatial  aspect  appropriate  to  individual 
perspectives  and  locations,  regardless  of  the  IG  type(s)  employed  at  each  location. 

•  Define  standards  to  enable  credible  rendering  of  weather  effects  on  sensor  performance  for  each  particular 
training  environment’s  point  of  view. 

•  For  laser  pointer  and  designator  beams,  define  methodologies  to  credibly  simulate  when  using  real  NVGs. 
Also  define  methodologies  to  render  credible  sparkle/dust/  attenuation  effects  in  the  atmosphere. 

•  Investigate  and  define  the  required  precision  and  associated  technical  means  of  simultaneously  tracking, 
with  six  degrees  of  freedom,  the  individual  locations  and  positions  of  multiple  physically  emulated  devices 
within  large  working  volumes.  In  parallel,  define  the  bandwidth  and  range  requirements  for  transmission 
of  signals  to  and  from  all  physically  emulated  devices  used  in  the  training  environment.  Consider  both 
information  security  requirements  and  unobtrusiveness  for  physical  fidelity  purposes.  Also  consider  both 
indoor  standalone,  indoor  networked  and  outdoor  training  environments  with  LVC  in  DMO. 
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Research  needs  and  opportunities  summary,  continued: 

•  For  bright  point  light  sources,  define: 

The  minimum  number  that  simultaneously  need  to  appear  in  a  scene  for  credible  representation. 
This  will  vary  depending  on  domain  and  scenario,  and  the  domain/scenario  having  the  largest 
number  as  its  minimum  requirement  is  the  quantity  of  interest.  And  thus  by  extension,  the 
minimum  required  quantity  of  point  source  engines. 

With  a  raster  scan  approach,  whether  galvanometer  speed  is  sufficient  to  avoid  distracting  flicker 
effects. 

With  a  raster  scan  approach,  whether  eye-safe  lasers  provide  sufficient  energy  at  any  point  on  the 
screen  to  induce  gain  variation  and  halo  effects  in  NVGs. 

Perceived  fidelity,  both  unaided  and  NVG-aided,  comparing  vector  and  raster  scan  approaches. 
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